Method and system for executing a containerized stateful application on a stateless computing platform using machine learning

ABSTRACT

A technique for executing a containerized stateful application that is deployed on a stateless computing platform is disclosed. The technique involves deploying a containerized stateful application on a stateless computing platform and executing the stateful application on the stateless computing platform. The technique also involves during execution of the stateful application, evaluating, in an application virtualization layer, events that are generated during execution of the stateful application to identify events that may trigger a change in state of the stateful application and during execution of the stateful application, updating a set of storage objects in response to the evaluations, and during execution of the stateful application, comparing events that are generated by the stateful application to the set of storage objects and redirecting a storage object that corresponds to an event to a persistent data store if the storage object matches a storage object in the set of storage objects.

FIELD OF THE INVENTION

The invention relates generally to executing a containerized statefulapplication on a stateless computing platform. For example, theinvention relates to capturing state information related to thecontainerized stateful application in a persistent data store while theapplication executes on the stateless computing platform.

BACKGROUND

Techniques for capturing the state of an application executing on astateless computing platform are known. One conventional technique forcapturing the state of an application executing on a stateless computingplatform involves monitoring a static set of storage objects to capturestate changes, where storage objects include, for example, state relatedfiles, directories, and registries. For example, a static set of filepaths and registries is predefined during a containerization phase andthe static set of file paths and registries is monitored in anapplication virtualization layer while the application is executed on astateless computing platform during a deployment phase. Although using astatic set of storage objects works well to capture state information,the technique will only capture state information for the storageobjects in the static set of storage objects.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates an example high-level workflow related to acontainerized stateful application that is executed on a statelessplatform in a manner that maintains the state of the containerizedstateful application.

FIG. 2 is a process flow diagram of a state capturing workflow for acontainerized stateful application that corresponds to the workflowillustrated in FIG. 1 and includes a containerization phase and adeployment phase.

FIG. 3 illustrates an example of a conventional technique for capturingthe state of a containerized application executing on a statelesscomputing platform during the deployment phase.

FIG. 4 depicts an example of a static set of storage objects that can bemonitored by an application virtualization layer to redirect stateinformation to a persistent data store.

FIG. 5 illustrates an example of a technique for executing acontainerized stateful application that is deployed on a statelesscomputing platform in which the capturing of state information isprimarily implemented within the application virtualization layer.

FIGS. 6A-6D depict a time sequence of a set of storage objects as theset of storage objects is updated on-the-fly as the containerizedapplication executes on the stateless computing platform.

FIGS. 7A and 7B are process flow diagrams of an example embodiment ofoperations that occur during the deployment phase of a containerizedstateful application.

FIG. 8 is a timing diagram of the workflow described above withreference to FIGS. 1, 2, and 5-7B.

FIG. 9 illustrates an example of a technique for executing acontainerized stateful application that is deployed on a statelesscomputing platform in which a portion of the state capturing workload isoffloaded to a state capturing service.

FIG. 10 illustrates operations corresponding to the techniqueillustrated in FIG. 9 that are performed by the applicationvirtualization layer during the deployment phase.

FIGS. 11A-11D illustrate operations corresponding to the techniqueillustrated in FIG. 9 that are performed by the state capture service.

FIG. 12 is a timing diagram of the workflow described above withreference to FIGS. 1, 2, and 9-11D.

FIG. 13 is an example of a generic frame format used for recording statechange operations.

FIG. 14 is an example of a create file/open file frame format.

FIG. 15 is an example of a write file frame format.

FIG. 16 is an example of a close file frame format.

FIG. 17 is an example of a create registry frame format.

FIG. 18 is an example of an open registry frame format.

FIG. 19 is an example of a modify registry frame format.

FIG. 20 is an example of a close registry frame format.

FIG. 21 is a process flow diagram of a method for executing acontainerized stateful application that is deployed on a statelesscomputing platform.

Throughout the description, similar reference numbers may be used toidentify similar elements. Additionally, in some cases, referencenumbers are not repeated in each figure in order to preserve the clarityand avoid cluttering of the figures.

DETAILED DESCRIPTION

It will be readily understood that the components of the embodiments asgenerally described herein and illustrated in the appended figures couldbe arranged and designed in a wide variety of different configurations.Thus, the following more detailed description of various embodiments, asrepresented in the figures, is not intended to limit the scope of thepresent disclosure, but is merely representative of various embodiments.While the various aspects of the embodiments are presented in drawings,the drawings are not necessarily drawn to scale unless specificallyindicated.

The described embodiments are to be considered in all respects only asillustrative and not restrictive. The scope of the invention is,therefore, indicated by the appended claims rather than by this detaileddescription. All changes which come within the meaning and range ofequivalency of the claims are to be embraced within their scope.

Reference throughout this specification to features, advantages, orsimilar language does not imply that all of the features and advantagesthat may be realized with the present invention should be or are in anysingle embodiment. Rather, language referring to the features andadvantages is understood to mean that a specific feature, advantage, orcharacteristic described in connection with an embodiment is included inat least one embodiment. Thus, discussions of the features andadvantages, and similar language, throughout this specification may, butdo not necessarily, refer to the same embodiment.

Furthermore, the described features, advantages, and characteristics ofthe invention may be combined in any suitable manner in one or moreembodiments. One skilled in the relevant art will recognize, in light ofthe description herein, that the invention can be practiced without oneor more of the specific features or advantages of a particularembodiment. In other instances, additional features and advantages maybe recognized in certain embodiments that may not be present in allembodiments of the invention.

Reference throughout this specification to “one embodiment,” “anembodiment,” or similar language means that a particular feature,structure, or characteristic described in connection with the indicatedembodiment is included in at least one embodiment. Thus, the phrases “inone embodiment,” “in an embodiment,” and similar language throughoutthis specification may, but do not necessarily, all refer to the sameembodiment.

The below described technique relates to a containerized statefulapplication that is deployed to an application portal for execution on astateless computing platform in a manner that maintains the state of thecontainerized stateful application even though the application isexecuted on a stateless computing platform.

FIG. 1 illustrates an example high-level workflow in a system 100related to a containerized stateful application that is executed on astateless platform in a manner that maintains the state of thecontainerized stateful application. The workflow includes operationsrelated to a containerizer 102, operations related to an applicationportal 104, operations related to a stateless computing platform 106,and operations related to a persistent data store 108.

The containerizer 102 includes a computing system that is used tocontainerize an application. Applications may include officeproductivity applications such as word processors, spreadsheets, andpresentation applications, application plug-ins, communicationsapplications such as web browsers and email clients, and a multitude ofenterprise applications, e.g., accounting, personnel management, sales,product management, project management, etc. In an embodiment,applications include applications configured to run on the WINDOWSoperating system. In an embodiment, applications may work in tandem witheach other. In another embodiment, an application may be formed by acombination of applications that work in tandem with each other. Forexample, a web browser plug-in may work in tandem with a web browser andthe combination of the web browser plug-in and the web browser isreferred to as an application.

In an embodiment, application containerization is an operating systemlevel virtualization method for deploying and running applicationswithout launching an entire virtual machine (VM) for each instance ofthe application. Instead, multiple isolated systems are run on a singlecontrol host and access a single kernel. Thus, a containerizedapplication is an application that includes all of the components suchas files, environment variables, and libraries necessary to run adesired application. In an embodiment, an application container(referred to herein as an “app container”) wraps an application in acomplete file system that contains all the components needed to runsuccessfully, including, for example, executable code, runtimecomponents, system tools, and system libraries. Using such an appcontainer, an application is able to run on many different platforms,including, for example, physical computers, virtual machines, bare-metalservers, and cloud server clusters.

Containerizing an application is the process of binding the relevantfiles and components to build an app container. In an embodiment,containerizing an application involves a containerizer engine 118 thatcaptures an application's install time (e.g., all files and registriesthat an application modifies while installing) and runtime dependenciesto generate a self-contained app container. The containerizer engine mayalso analyze the application's runtime environment to detectunidentified files/registries and/or read/write operations that areperformed by the application at runtime. The resulting app containerincludes the files and registries along with install time rules. Forexample, install time rules include file and registry redirection rules.In an embodiment, the install time rules of a particular app package arethe same for every instance of the application that is downloaded fromthe application portal by a user for execution on the statelesscomputing platform. Thus, the install time rules are agnostic to aparticular user.

In general, application virtualization is a technique in which anapplication is decoupled from the underlying operating system of thephysical machine (or machines) on which the application executes.Various techniques for application virtualization are known in thefield. In general, a “virtualized application” or “applicationvirtualization” involves techniques that decouple applications from theunderlying operating system of the computing platform. In an embodiment,decoupling applications from the underlying operating system of thecomputing platform involves separating, dividing, insulating, and/orprotecting the operating system from the virtualized application. Forexample, an application is virtualized so that when the application isinstalled on a computing platform, the application does not write ordilute the operating system files or registries and/or does not corruptthe operating system itself. In an embodiment, an applicationvirtualization layer creates a “sandbox,” which is an insulatedenvironment on the computing platform. For example, a “sandbox” mayinclude controls that limit access by the application to files,registries, certain hardware, and/or certain other applications.Application virtualization can utilize user-space/user-mode (referred toherein as “user-space”) virtualization techniques and/orkernel-space/kernel-mode (referred to herein as “kernel-space”)virtualization techniques. In an embodiment, user-space virtualizationcan be implemented without needing administrative rights to accesscertain resources, such as certain files/registries. In an embodiment, avirtualized application is not installed on the computing platform in aconventional manner in that in a convention installation the applicationdeclares, while installing on the operating system, that the applicationis installing and hence the operating system is aware that something wasinstalled. Additionally, in a conventional installation, the applicationwrites specific files/registries in the installation areas of theregistry (e.g., protected areas of the memory that require admin rightsto access) to make the operating system aware of the application. On theother hand, in an embodiment, a virtualized application does not declareto the operating system that the application is installing. Therefore,the operating system does not necessarily know that an application wasinstalled at least because the application writes all the files andregistries used by the application to run in a virtualized file systemand a virtualized registry area. In an embodiment, a virtualized filesystem and virtualized registry are files and registries that arecreated in a separate location (e.g., a separate namespace) other thanthe operating system's default location. Locating such files andregistries in a separate location (e.g., in a separate namespace) is atechnique used to sandbox, or “containerize,” the files and/orregistries used by the application. Using the virtualized file systemand virtualized registry, the operations triggered by the applicationwill be performed on the virtual files and/or registries to prevent theapplication from reading/writing to the system folders and registries ofthe operating system. For example, an application might require itsfiles to be in “c:\ProgramFiles\ApplicationVendor\ApplicationName.” Inan embodiment, these files are virtualized so that when the applicationlooks for files in “c:\ProgramFiles\ApplicationVendor\ApplicationName”the file system API calls are redirected to“c:\Sandbox\ProgramFiles\ApplicationVendor\ApplicationName.”

An application that is installed in a conventional matter is oftenreferred to as a “native” application whereas an application installedusing a virtualized file system and a virtualized registry area isreferred to as a “virtualized application.” In an embodiment, executionof a virtualized application at run-time is controlled by an applicationvirtualization layer. In an embodiment, a virtualized applicationappears to a user to interface directly with the computing platform'soperating system, but the virtualized application interfaces with theapplication virtualization layer. In an embodiment, the applicationvirtualization layer intercepts communications (e.g., events such as APIcalls) between the application and the operating system and acts as aproxy between the application and the native system registries andsystem files of the operating system running on the computing platform.In an embodiment, the application virtualization layer “hooks” intoevents (e.g., API calls) at the dynamic-link library (DLL) (e.g., NTDLL)level. Implementing application virtualization using hooking in theapplication virtualization layer is known in the field. In anembodiment, the application virtualization layer can implement virtualregistry files and secure file stores within the sandbox.

Referring again to FIG. 1, the application portal 104 is a computingresource that stores an app container 110 for access by an end-user 112via an end-user device 114. In an embodiment, the application portal maystore more than one containerized application. For example, in anembodiment, an enterprise may maintain an application portal thatincludes multiple different containerized applications that can beaccessed by end-users on an as-needed basis. In an embodiment, access tothe applications is subject to proper access rights. In an embodiment,the containerized application portal includes a web based architecture,which could be hosted in a public and/or private cloud or on-premise.The application portal is capable of saving the app containers, savingapplication-specific and user-specific state information (app state 120)in a persistent data store, handling user/group based roles for each appcontainer, performing on-the-fly customizations, and delivering the appcontainers on-demand from, for example, a public/private cloud oron-premise servers.

The stateless computing platform 106 includes computing resources thatcan be used to execute the containerized applications that are stored onthe application portal. In an embodiment, the stateless computingplatform may include computing resources such as a stateless pool ofservers, stateless Remote Desktop Service (RDS) pools, a server-lesscomputing architecture, an Infrastructure as a service (IaaS)environment, desktop systems, physical or virtual disks, system volumes,application layering disks, etc. In an embodiment, the computingplatform is “stateless” in that state information generated duringexecution of the containerized application is not saved in the memory ofthe stateless computing platform once the application is closed, e.g.,the state information does not “persist” in the stateless computingplatform after execution of the application is ended.

In an embodiment, the persistent data store 108 includes memory to storestate information related to execution of an application so that thestate information survives after the process with which the data wascreated has ended, e.g., after the application is closed or ended. Thepersistent data store may include non-volatile memory such assemiconductor or solid state memory (e.g., flash memory), magnetic tape,a rigid magnetic disk, and an optical disk. In an embodiment, thepersistent data store may be implemented as a volume storage, networkshare storage, and/or cloud storage. The persistent data store may beintegrated within the application portal 120 and/or separate from theapplication portal.

The workflow illustrated in FIG. 1 is now described with reference to asequence of task arrows that are identified as task arrows T1-T8. Taskarrow T1 illustrates the generation of an app container 110 by thecontainerization engine 118 of the containerizer 102. In an embodiment,the containerization engine generates the app container by capturing anapplication's install time (e.g., using snapshots to capture all filesand registries that an application modifies while installing) andruntime dependencies to generate a self-contained app container. In anembodiment, the app container includes a complete file system thatcontains all the components needed for the application to runsuccessfully, including, for example, executable code, runtimecomponents, system tools, an application virtualization layer, andsystem libraries.

Task arrow T2 illustrates the uploading of the app container 110 to theapplication portal 104, where the app container is stored for use by anend-user 112. Once the app container is uploaded to the applicationportal, the containerized application is considered to be “deployed” forgeneral use. In one embodiment, general use may be limited to userswithin an enterprise, in another embodiment, general use may be limitedto users with certain access rights (e.g., registered users,subscribers, etc.), and in another embodiment, general use may includeany user that has access to the public Internet. Identifying the breathof access to a containerized application that is deployed on theapplication portal is implementation specific. Tasks T1 and T2 aretypically performed by a system administrator, such as an InformationTechnology (IT) specialist.

Task arrow T3 illustrates deployment of the containerized applicationonto the stateless platform for execution in response to activation bythe end-user 112 that accesses the application portal via the end-userdevice 114. Execution of the containerized application involvesdownloading of the app container 110 to the stateless computing platform106, unpacking of the app container, and execution of the application onthe stateless computing platform. Operations related to this deploymentare identified as “execution instance 1,” 130. Execution instance 1 mayutilize any of the computing resources available in the statelesscomputing platform. For example, execution instance 1 may use a firstset of computing resources (e.g., virtual machines, servers, CPUs, RAM,etc.) to execute the containerized application. In an embodiment, theend-user device may be a computing device that enables access to theapplication portal via a network connection 134, which could be a wiredconnection, a wireless connection, or a combination thereof. Exampleend-user devices include desktop computers, laptop computers, thinclient computers, pad computers, and mobile devices such as smartphonesand smart watches. Other types of end-user devices are also possible.

Upon executing the containerized application on the stateless computingplatform 106, a first operation may involve checking to see if there isany state information that has been previously stored for the particularcontainerized application and the particular end-user of thecontainerized application. In an embodiment, checking for stateinformation is controlled by a deployment engine (not shown) thatexecutes upon selecting the containerized application. Task arrow T4illustrates previously captured state information (e.g., app state 120)being obtained by the containerized application from the persistent datastore for execution instance 1.

Once the previously captured state information (e.g., app state 120) isobtained by the containerized application from the persistent data store108, the application is executed in execution instance 1 130 ascontrolled by the end-user 112. For example, the end-user may interactwith/control a productivity application or a database application fromthe end-user device 114. During execution of the containerizedapplication, state information may be generated by the containerizedapplication. For example, state information may include storage objectssuch as configuration files, configuration databases, directories thatinclude configuration information, registries, and information generatedduring execution of the application. For example, during execution of anapplication new records (e.g., new account records such as new customerrecords, new report templates, new invoices, etc.) may be generatedand/or modified that are also included in the state information. Whenusing a stateful application, it is desirable to store such stateinformation so that the end-user will have continuity between uses ofthe application. For example, if an end-user sets a user preference(e.g., a preferred font) in the application, it is desirable to have theuser preference stay set the next time the application is accessed fromthe application portal 104 by the user. Task arrow T5 illustrates stateinformation of the application being stored in the persistent datastore. Techniques for capturing and storing state information in thepersistent data store are described in detail below. In an embodiment,closing the application will trigger saving of state information to thepersistent data store. In an embodiment, state information is stored tothe persistent data store on a periodic basis during use of theapplication (e.g., every 5 minutes) and upon closing the applicationalthough other approaches are possible. For example, the stateinformation may be stored only upon closing the application.

After an end-user has closed an application, the end-user may at somepoint in time use the containerized application again. For example, theend-user 112 may navigate to the application portal 104 and select thedesired containerized application or the user may activate a shortcuticon available on the user's device. In any case, task arrow T6illustrates a second deployment of the containerized application ontothe stateless computing platform 106 for execution. Again, execution ofthe containerized application involves downloading of the app container110, unpacking of the app container, and execution of the application onthe stateless computing platform. Operations related to the seconddeployment are referred to as “execution instance 2,” 132. Executioninstance 2 may utilize any of the computing resources (e.g., virtualmachines, servers, CPUs, RAM, etc.) in the stateless computing platform.For instance, execution instance 2 may use a second set of computingresources to execute the application and the set of computing resourcesused in execution instance 2 may be different from the set of computingresources used in execution instance 1. Although not depicted in FIG. 1,it is also possible that execution instance 2 could execute on adifferent stateless computing platform. That is, the app container 110could be downloaded to computing resources of a stateless computingplatform that is separate from the stateless computing platform 106,such that execution instances of the containerized application can roamfrom instance to instance between stateless computing platformsdepending on conditions such as, for example, resource load, resourcecapability, and resource cost.

Upon executing the containerized application at the stateless computingplatform 106, a first operation may involve checking to see if there isany state information that has been previously stored for the particularcontainerized application and the particular user of the application.Task arrow T7 illustrates previously stored state information (e.g., appstate 120) being obtained by the containerized application from thepersistent data store 108 for execution instance 2, 132.

Once the stored state information is obtained by the containerizedapplication from the persistent data store 108, the application isexecuted in execution instance 2 as controlled by the end-user 112. Forexample, the end-user interacts with/controls the same productivityapplication or database application as was used in execution instance 1.During execution of the application, additional state information may begenerated by the containerized application. When using a statefulapplication, it is desirable to store such state information so that theend-user will have continuity between uses of the application. Forexample, if an end-user sets another user preference (e.g., a preferredtoolbar layout) in the application, it is desirable to have the userpreference stay set the next time the application is accessed by theuser from the application portal. Task arrow T8 illustrates stateinformation of the application being stored in the persistent datastore. Again, the techniques for capturing and storing state informationin the persistent data store are described in detail below. In anembodiment, closing the application triggers saving of the stateinformation to the persistent data store. In an embodiment, stateinformation is stored to the persistent data store on a periodic basisduring use of the application (e.g., every 5 minutes) and upon closingthe application.

As can be seen from the workflow illustrated in FIG. 1, applicationstate can be maintained on a per-application and per-user basis acrossmultiple uses of the particular containerized application by aparticular end-user even though the containerized application isexecuted on a stateless computing platform. FIG. 2 is a process flowdiagram of a state capturing workflow for a containerized statefulapplication that corresponds to the workflow illustrated in FIG. 1. Theprocess flow illustrated in FIG. 2 can be generally categorized into twophases, a containerization phase 200 and a deployment phase 202.Operations of the containerization phase include containerizing theapplication (block 210), capturing the application installation state(block 212), and uploading the containerized application (e.g., as anapp container) to the application portal (block 214). In an embodiment,containerizing the application involves capturing an application'sinstall time (e.g., using snapshots to capture all files and registriesthat an application modifies while installing) and runtime dependenciesto generate a self-contained app container. In an embodiment, the appcontainer includes a file system that contains all the applicationcomponents needed to run the application successfully, including, forexample, the application executables, runtime components, system tools,a virtualization layer, and system libraries. Tasks T1 and T2 in FIG. 1correspond to the containerization phase. In an embodiment, capturingthe application's installation state involves capturing theapplication's install time rules, e.g., the file and registryredirection rules. In an embodiment, capturing the installation state ofan application involves determining which files and/or registries havechanged after the application is installed. In an embodiment, theinstallation state is captured by taking a pre-installation snapshot ofthe files and registries, installing the application, and then taking apost-installation snapshot of the files and registries to determine whathas changed. The difference between the files and registries between preand post installation constitutes the “installation state.” In anembodiment, containerizing an application involves creating aconfiguration file with the redirection rules for the files and/orregistries of the installation state and copying the configuration fileinto the app container, which can be uploaded to the application portal.

In an embodiment, uploading the containerized application involvesuploading the app container to the application portal for general use bythe appropriate end-users. In an embodiment, operations of thecontainerization phase 200 are performed by a system administrator, suchas an IT specialist. Once the app container is uploaded to theapplication portal, the containerized application is considered to be“deployed” for general use by end-users. As described above, access tocontainerized applications on the application portal is implementationspecific.

Operations of the deployment phase 202 include downloading the appcontainer from the application portal to the stateless computingplatform (block 216), synchronizing application state (block 218),capturing application state (block 220), and saving application state(block 222). The deployment phase may be implemented at least in part bythe application portal 104 and the stateless computing platform 106 asdescribed above with reference to FIG. 1. In an embodiment, an end-usernavigates to the application portal and triggers (e.g., by doubleclicking on an application icon) execution of the containerizedapplication on the stateless computing platform. For example, doubleclicking on an application icon selects the containerized applicationand causes the app container to be downloaded to the stateless computingplatform for execution. In an embodiment, a next operation involvessynchronizing the application's state with state information (e.g.,application-specific and user-specific state information) that waspreviously stored in the persistent data store. For example,synchronizing application state may involve accessing a state trackingservice at the application portal to find state information related tothe particular application and end-user combination. In an embodiment,double clicking on the app container launches a deployment engine thatmanages downloading of the application container and synchronizing theapplication's state.

In an embodiment, a next operation involves capturing the state of theapplication as the application executes on the stateless computingplatform (block 220). A technique for capturing the state of theapplication as the application executes on the stateless computingplatform is described in detail below with reference to FIGS. 5-21.

In an embodiment, a next operation is to save the application's state inthe persistent data store (block 222). For example, the application'sstate is saved on an application-specific and a user-specific basis sothat the same end-user of a containerized application will experienceconsistent state between subsequent uses of the same containerizedapplication. In an embodiment, state information is stored to thepersistent data store on a periodic basis during use of the application(e.g., every 5 minutes) and upon closing the application.

Techniques for capturing the state of an application executing on astateless computing platform are known. One conventional technique forcapturing the state of an application executing on a stateless computingplatform involves monitoring a static set of storage objects to capturestate changes, where storage objects include, for example, state relatedfiles, directories, and registries. For example, a static set of filepaths and registries is predefined during the containerization phase andthe static set of file paths and registries is monitored in anapplication virtualization layer while the application is executed on astateless computing platform during the deployment phase. FIG. 3illustrates an example of a conventional technique for capturing thestate of a containerized application executing on a stateless computingplatform during the deployment phase. As illustrated in FIG. 3, acontainerized application 140 and an application virtualization layer144 execute in the user-space 146 of the stateless computing platform106 and stateless computing platform resources 156 execute in thekernel-space 148. During execution of the containerized application,events are generated, for example, the events include API calls as isknown in the field and the events are monitored by the applicationvirtualization layer. In an embodiment, virtualization layer logic 150of the application virtualization layer monitors events that aregenerated by the containerized application to determine if an eventmatches a storage object in the static set of storage objects 152. Forexample, the virtualization layer logic compares file paths orregistries of API calls to the static set of file paths and registriesto see if there is a match. In an embodiment, the set of storage objectsthat is monitored is static in that the set of storage objects does notchange during the deployment phase. FIG. 4 depicts an example of astatic set of storage objects that can be monitored by an applicationvirtualization layer to redirect state information to a persistent datastore. During operation, when an event matches a storage object in thestatic set of storage objects, then state changes related to the storageobject are redirected to the persistent data store 108. FIG. 3illustrates state changes that have been redirected to the persistentdata store by the application virtualization layer. If an event does nothave a file path or registry that matches a file path or registry in thestatic set of storage objects, then the event is determined to benon-state related and the event passes to the stateless computingplatform resources for processing. FIG. 3 also illustrates non-staterelated events being passed from the application virtualization layer tostateless computing platform resources 156 of the stateless computingplatform for processing.

Although the technique illustrated in FIG. 3 works well to capture stateinformation corresponding to the static set of storage objects 152 thatis monitored by the application virtualization layer 144, the techniquewill only capture state information for the storage objects in thestatic set of storage objects. Therefore, if the static set of storageobjects does not include every possible storage object that may includestate information, some state information may not be captured in thepersistent data store. Many applications utilize a wide variety ofstorage objects depending on the manner in which the applications areused. Trying to identify every possible state-related storage objectduring the containerization phase for every containerized applicationthat may be offered on an application portal can be a difficult andresource intensive operation. For example, identifying state relatedstorage objects during the containerization phase 200 (FIG. 2) istypically a manual process that requires constant human interaction toinfluence workflow and manually identify the corresponding state relatedstorage objects. Even during the manual process, it is quite difficultto ensure that all state related storage objects are identified.

In accordance with an embodiment of the invention, a technique forexecuting a containerized stateful application that is deployed on astateless computing platform is disclosed. The technique involvesdeploying a containerized stateful application on a stateless computingplatform and executing the stateful application on the statelesscomputing platform. The technique also involves during execution of thestateful application, evaluating, in an application virtualizationlayer, events that are generated during execution of the statefulapplication to identify events that may trigger a change in state of thestateful application and during execution of the stateful application,updating a set of storage objects in response to the evaluations, andduring execution of the stateful application, comparing events that aregenerated by the stateful application to the set of storage objects andredirecting a storage object that corresponds to an event to apersistent data store if the storage object matches a storage object inthe set of storage objects. As indicated above, the technique involvesevaluating events that may trigger a change in state of the statefulapplication and updating a set of storage objects in response to theevaluation. Because the set of storage objects is updated duringexecution of the application, the technique does not rely on a staticset of storage objects to capture all state information. It has beenrealized that there are certain events generated by a containerizedstateful application that tend to trigger changes in state. For example,it has been learned that certain file related API calls with a desiredaccess of “write” and certain registries with a desired access of“write,” “set,” or “set value” tend to correspond to changes in state ofthe executing stateful application. Thus, by configuring an applicationvirtualization layer to identify such events as a containerizedapplication is executing on a stateless computing platform, a set ofstorage objects can be generated and updated during the deployment phaseas the containerized application generates events that are specific to aparticular end-user's interaction with and/or control of thecontainerized application. That is, a set of storage objects isautomatically learned (e.g., by “machine learning”) during execution ofthe application in the deployment phase without the need for humaninteraction and without having to manually run through each possibleworkflow in the containerization phase. Applying machine learning to theprocess of capturing state information for a containerized applicationduring the deployment phase enables a set of storage objects to becustom generated “on-the-fly” on a per-application and per-user basis ina manner that can be efficiently implemented. In addition, because stateinformation is captured and stored in a persistent data store on-the-flyon a per-application and per-user basis, a containerized application caneasily roam between stateless computing platforms, resulting in a“nomadic” containerized application or simply a nomadic container.

For example, in WINDOWS applications, it has been found that certainfile related events with the desired access of “write” tend to relate tostate information. In particular, events identified at least in part as“CreateFile” and “WriteFile” with a desired access of “write” tend torelate to state information. Additionally, it has been found thatcertain registry related events with desired access of “write,” “set,”or “set value,” tend to relate to state information. In particular,events identified at least in part as “RegCreateKey,” “RegOpenKey,”“RegSetKeyValue,” and “RegSetValue” tend to relate to state information.With such events identified, the application virtualization layer of acontainerized application can be configured to recognize these eventswhile the application is being used by an end-user and to trigger anautomatic update of the set of storage objects during execution of theapplication on the stateless computing platform.

The technique for capturing state information can be implemented indifferent ways. For example, in one embodiment, the capturing of stateinformation is implemented primarily within the applicationvirtualization layer (referred to below as the “dynamic redirection”approach). In another embodiment, a portion of the state capturingworkload is offloaded to a state capturing service (referred to below asthe “state capture service” approach), which can reduce the processingburden on the application virtualization layer.

FIG. 5 illustrates an example of a technique for executing acontainerized stateful application that is deployed on a statelesscomputing platform in which the capturing of state information isprimarily implemented within the application virtualization layer. Asillustrated in FIG. 5, a containerized application 140 (e.g., deployedfrom an application portal) and an application virtualization layer 160,which may be included in the application container or downloaded fromthe application portal by a deployment engine, execute in the user-space146 of a stateless computing platform 106. During execution of thecontainerized application, events are generated, for example, the eventsinclude API calls as is known in the field. The events are monitored bythe application virtualization layer. In an embodiment, virtualizationlayer logic 162 of the application virtualization layer evaluates theevents that are generated by the containerized application to determineif an event matches an identified set of events with a particulardesired access. For example, the virtualization layer logic monitors theevents for events that include a particular call type with a desiredaccess of “write,” “set,” or “set value.” In an embodiment, thevirtualization layer logic monitors for events such as CreateFile,WriteFile, DeleteFile, CloseFile, RegCreateKey, RegOpenKey,RegSetKeyValue, RegSetValue, RegSaveKey, RegDeleteKey, andRegDeleteValue events or events with similar functionality in order tocapture state information. If such events are identified, the events areevaluated to see if a set of storage objects should be updated. Forexample, if a CreateFile event includes a desired access of “write,”then a storage object (e.g., C:\pathA\pathB\fileC.txt) that correspondsto the event should be added to the set of storage objects if thestorage object does not already exist in the set of storage objects. Inan embodiment, storage objects may include configuration files (e.g.,files that include path redirection rules), directories that includeconfiguration information, configuration databases, data files, andother files and/or registries that relate to the state of anapplication.

With reference to FIG. 5, in an embodiment, the virtualization layerlogic 162 filters the events to identify events with a desired access of“write,” “set,” or “set value.” In an embodiment, “set” and “set value”are considered synonymous. When identified, such events are handled by amonitored set update engine 164 of the application virtualization layer160. In an embodiment, the monitored set update engine is implemented incomputer executable code that is integrated into the applicationvirtualization layer and the monitored set update engine evaluates anevent to determine if a storage object corresponding to the eventmatches a storage object that is already in a set of storage objects 168that is held in the in-process memory of the application virtualizationlayer. In an embodiment, in-process memory refers to memory of thestateless computing platform that is cleared (i.e., does not persist)after an execution instance ends. For example, volatile memory used toexecute an application in a particular execution instance is “in-processmemory” that is cleared in the stateless computing platform onceexecution of the application is ended. If the storage objectcorresponding to the event does not match a storage object that isalready in the set of storage objects, then the storage object is addedto the set of storage objects. FIGS. 6A-6D depict a time sequence of aset of storage objects as the set of storage objects is updatedon-the-fly as the containerized application executes on the statelesscomputing platform. FIG. 6A depicts the set of storage objects at time,t₁. At some point later in time, the set of storage objects is updatedin response to an event that was generated by the containerizedapplication executing on the stateless computing platform. FIG. 6Bdepicts the set of storage objects at time, t₂. At some point later intime, the set of storage objects is updated in response to an event thatwas generated by the containerized application executing on thestateless computing platform. FIG. 6C depicts the set of storage objectsat time, t₃. At some point later in time, the set of storage objects isupdated in response to an event that was generated by the containerizedapplication executing on the stateless computing platform. FIG. 6Ddepicts the set of storage objects at time, t₄. In an embodiment, theset of storage objects (identified as a “dynamic” set of storage objectsin FIG. 5) is maintained in in-process memory of the applicationvirtualization layer as, for example, a search tree that is used by thevirtualization layer logic to monitor storage objects. Because thecontainerized application is executed on the stateless computingplatform, the set of storage objects that is maintained in thein-process memory of the application virtualization layer will not bemaintained in the in-process memory of the application virtualizationlayer once the application is ended.

In addition to updating the set of storage objects 168 in the in-processmemory of the application virtualization layer 160, the monitored setupdate engine 164 may periodically store the set of storage objects 170,or updates thereof, in the persistent data store 108. Storing the set ofstorage objects in the persistent data store enables the set of storageobjects to be recalled upon a subsequent use of the application by thesame end-user. Thus, the set of storage objects is maintained betweenuses (i.e., persists) and can be recalled into in-process memory of thestateless computing platform upon a subsequent use of the containerizedapplication 140, which can reduce the processing load on the applicationvirtualization layer to avoid delays that may be attributed to statecapturing operations.

In addition to updating the set of storage objects 168 that is monitoredby the application virtualization layer 160, the applicationvirtualization layer also redirects state related storage objects to thepersistent data store 170. For example, storage objects corresponding toevents that tend to trigger changes in state are redirected to thepersistent data store by the application virtualization layer usingapplication virtualization techniques. FIG. 5 illustrates stat changesbeing sent from the application virtualization layer to the persistentdata store to update application-specific and user-specific stateinformation 172. State information can be stored in the persistent datastore on an application-specific and user-specific basis so that a usercan have a consistent user experience between uses of the containerizedapplication.

The operations described with reference to FIGS. 5-6D occur during thedeployment phase 202 (FIG. 2) of a containerized stateful application.FIGS. 7A and 7B are process flow diagrams of an example embodiment ofoperations that occur during the deployment phase of a containerizedstateful application. At block 702, the containerized application islaunched. For example, launching the containerized application involvesan end-user navigating to an application icon at an application portaland then selecting (e.g., double clicking on) the application icon. Atblock 704, launching the application causes the corresponding appcontainer to be downloaded to the stateless computing platform,unpackaged, and executed. At block 706, there is a search for stateinformation (application state) that is stored in the persistent datastore for the user launching the containerized application. In anembodiment, a deployment engine, which may be triggered upon selectionof the app container, may search a persistent data store forapplication-specific and user-specific state information that matchesthe current application and the end-user. In an embodiment, a deploymentengine is implemented as user-space computer code that runs on thestateless computing platform and supports downloading the app containerfrom the application portal, synchronizing the application's state fromthe persistent data store, preparing the environment for the applicationto run, and maintaining offline allowance, licensing, and usagereporting on a per-application and per-user basis.

At block 708, if state information has been previously stored in apersistent data store, the state information (application state) isdownloaded from the persistent data store and incorporated into theapplication virtualization layer. For example, application-specific anduser-specific state information is synchronized with the application andthe set of storage objects is loaded into the in-process memory of theapplication virtualization layer. Using such an approach, thecontainerized application that is stored on the application portal isagnostic to the end-user, but from the user's perspective, state ismaintained between uses of the application.

At block 710, the application is run on the stateless computingplatform. At block 712, the application virtualization layer redirectsstorage objects corresponding to certain events to the persistent datastore as specified by redirection rules of the applicationvirtualization layer (e.g., redirection rules provided in aconfiguration file).

At decision point 714, it is determined if the application is stillexecuting on the stateless computing platform. If the application isstill executing on the stateless computing platform, then the processproceeds (via element A) to decision point 716 (FIG. 7B). At decisionpoint 716, it is determined if the event is related to the state of theapplication. If the event is not related to the state of theapplication, then the process proceeds (via element B) back to block 712(FIG. 7A). If, however, the event is related to the state of theapplication (e.g., the event has a particular name and has a desiredaccess of “write,” “set,” or “set value”), then the process proceeds todecision point 718. At decision point 718, it is determined if thecorresponding storage object is included on an exclusion list. In anembodiment, an exclusion list identifies storage objects that do notrelate to state information and/or to storage objects that should beexcluded from the state capturing process for some other reason. If thestorage object is on the exclusion list, then the process proceeds (viaelement B) back to block 712 (FIG. 7A). Using an exclusion list may helpto prevent storage objects that do not relate to the state of anapplication from being added to the set of storage objects that ismonitored by the application virtualization layer. In an embodiment, theexclusion list can be used to exclude information from being persistedeven if the information is state related. The exclusion list can give auser the option to customize or “tune” the capture of state information.For example, an application may have forty files on which a writeoperation is performed but out of the forty files, a user may only wantto capture state information for ten of the files. Thus, thirty of thefiles could be added to the exclusion list to prevent state informationfrom being captured and saved in the persistent data store. If thestorage object is not on an exclusion list, then at block 720, thestorage object is added to the set of storage objects that is monitoredby the application virtualization layer. For example, the storage objectis added to a search tree that is held in the in-process memory of theapplication virtualization layer as the containerized applicationexecutes on the stateless computing platform. Thus, a set of storageobjects is automatically learned (e.g., by “machine learning”) duringexecution of the application in the deployment phase without the needfor human interaction and without having to manually run through eachpossible workflow in the containerization phase. Next, at block 722, thestorage object is updated in the persistent data store. For example, thestorage object is added to a configuration file that includes the set ofstorage objects. In an embodiment, the persistent data store is updatedupon identification of each new storage object and in other embodiments,updates are made periodically, in batches, and/or after the applicationis closed. Additional steps may be taken to make the state informationaccessible for subsequent uses. For example, at block 724, a pathnotification may be sent to a service hosted at the application portalto update the service on the application-specific and user-specificpath. At block 726, a resource (e.g., a storage object such as afile/registry) is created in a containerized location at, for example,the application portal. The process then returns to element A.

Returning back to decision point 714 in FIG. 7A, if it is determinedthat the application is no longer executing (e.g., the user has signaledto close the application), at block 728, the application's latest stateinformation is updated by sending the latest state information, or anylast state updates, to the persistent data store. In the process flowdiagrams of FIGS. 7A and 7B, state related workflow is indicated withinthe dashed line boxes 730.

FIG. 8 is a timing diagram of the workflow described above withreference to FIGS. 1, 2, and 5-7B. At operation 802, an app container isuploaded from the containerizer to the application portal. In FIG. 8,the application portal includes a persistent data store that is used tostore application state information. At operation 804, the app containeris downloaded from the application portal to a deployment engine thatexecutes on the stateless computing platform. In an embodiment, thedeployment engine is included in the app container and executes upondownloading of the app container. At operation 806, the state of theapplication is synchronized with any state information that is stored inthe persistent data store. At operation 808, the application is run(e.g., executed on the stateless computing platform) using theapplication virtualization layer. Once the application begins running,events are evaluated by the application virtualization layer to identifyevents that may trigger state changes and a set of storage objects isupdated in response to the evaluations. As described above, machinelearning is used to identify state related storage objects duringexecution of the application on the stateless computing platform and theidentified state related storage objects are recorded (e.g., added to aset of storage objects). The set of storage objects is stored in thein-process memory of the application virtualization layer while theapplication is executing on the stateless computing platform and the setof storage objects is also stored in the persistent data store so thatthe state information is preserved for a next use of the containerizedstateful application by the same user.

At operation 810, the application portal is notified of a state changeevent so that an application-specific and user-specific configurationfile can be updated to ensure that the state information (e.g.,redirection rules) is captured and persists after the execution instanceends. In an embodiment, the application portal updates theapplication-specific and user-specific configuration file in response tothe state change event notification. At operation 812, application stateinformation is recorded, e.g., passed from the in-process memory of theapplication virtualization layer to the deployment engine. At operation814, the deployment engine sends an application state update to thepersistent data store. At operation 816, the application is closed(e.g., execution of the application on the stateless computing platformis ended) and application state information is recorded, e.g., sent fromthe application virtualization layer to the deployment engine. Atoperation 818, the latest state information, or any last state update,is sent form the deployment engine to the persistent data store forstorage.

In the embodiment described above with reference to FIGS. 5-8, theapplication virtualization layer implements the operations to capturestate information and to update the set of storage objects. In anotherembodiment, a portion of the processing to capture the state informationis offloaded from the application virtualization layer to a statecapture service, for example, a state capture service that executesseparate from the application virtualization layer. FIG. 9 illustratesan example of a technique for executing a containerized statefulapplication that is deployed on a stateless computing platform in whicha portion of the state capturing workload is offloaded from theapplication virtualization layer to a state capturing service. Byoffloading some operations related to capturing state information fromthe application virtualization layer to a state capture service, theapplication virtualization layer is able to efficiently implementvirtualization operations with less drag from state capturing operationswhile still maintaining the state of the stateful application as thestateful application executes on the stateless computing platform. In anembodiment, the application virtualization layer identifies the eventsthat tend to trigger changes in state and then sends state relatednotifications (e.g., from a notification engine) to the state captureservice, which uses the notifications to capture application-specificand user-specific state information and to update a set of storageobjects. In an embodiment, the state capture service is embodied in codethat is not commingled with the code of the application virtualizationlayer and executes on the stateless computing platform as a separateinstance from the execution of the containerized application.

As illustrated in FIG. 9, a containerized application 140 (e.g.,deployed from an application portal), an application virtualizationlayer 180, which may be included in the application container, and astate capture service (SCS) 190 execute in the user-space 146 of astateless computing platform 106. During execution of the containerizedapplication, events are generated, for example, the events include APIcalls as is known in the field. The events are monitored by theapplication virtualization layer. In an embodiment, virtualization layerlogic 182 of the application virtualization layer evaluates events thatare generated by the containerized application to determine if an eventmatches an identified set of events with a particular desired access.For example, the virtualization layer logic monitors the events forevents that include a desired access of “write,” “set,” or “set value.”In an embodiment, the virtualization layer logic monitors for eventssuch as CreateFile, WriteFile, DeleteFile, CloseFile, RegCreateKey,RegOpenKey, RegSetKeyValue, RegSetValue, RegSaveKey, RegDeleteKey, andRegDeleteValue events.

If such events are identified, the events are provided to a notificationengine 184 that executes as part of the application virtualization layer180. The notification engine sends notifications to the state capturingservice 190 and the state capturing service evaluates the events to seeif the set of storage objects should be updated. For example, if aCreateFile event includes a desired access of “write,” then acorresponding storage object (e.g., C:\pathA\pathB\fileC.txt) of theevent should be added to a set of storage objects if the storage objectdoes not already exist in the set of storage objects.

With reference to FIG. 9, in an embodiment, notifications are handled bya monitored set update engine 194 of the state capturing service 190.The monitored set update engine evaluates the event (which iscommunicated via a notification) to determine if a storage objectcorresponding to the event matches a storage object that is already inthe set of storage objects 168. If the storage object corresponding tothe event does not match a storage object that is already in the set ofstorage objects, then the storage object is added to the set of storageobjects. The set of storage objects can be updated as described abovewith reference to FIGS. 6A-6D. In an embodiment, the set of storageobjects is maintained in in-process memory of the state capture serviceas, for example, a search tree that is used by the state capture serviceto monitor storage objects. Additionally, the monitored set updateengine may periodically store the set of storage objects 170, or updatesthereof, in the persistent data store 108. Storing the monitored set ofstorage objects in the persistent data store enables the set of storageobjects to be recalled for a subsequent use of the application by thesame end-user. Thus, the monitored set of storage objects is maintainedbetween uses (e.g., persists) and can be recalled into in-process memoryof the stateless computing platform upon a subsequent use of thecontainerized application, which can reduce the processing load on thestate capturing service to avoid delays that may be attributed to statecapturing operations.

In addition to updating the set of storage objects 168 that is monitoredby the state capture service 190, the state capture service alsoredirects state related storage objects to the persistent data store.For example, storage objects corresponding to events that tend totrigger changes in state are redirected to the persistent data store bythe state capture service using redirection rules obtained from aconfiguration file and redirection rules learned from processing thenotifications received from the notification engine 184. In anembodiment, state information 172 can be stored in the persistent datastore 108 on an application-specific and user-specific basis so that anend-user can have a consistent user experience between uses of thecontainerized application.

The operations described with reference to FIG. 9 occur during thedeployment phase 202 (FIG. 2). FIGS. 10-11D are process flow diagrams ofan example embodiment of operations that occur during the deploymentphase of a containerized stateful application. For example, FIG. 10illustrates operations that are performed by the applicationvirtualization layer during the deployment phase. At block 1002, anevent is received at, for example, a function handler, and evaluated. Inan embodiment, the event includes an API call and a storage object, forexample, the event may be: CreateFile(C:\pathA\pathB\fileC.txt), withdesired access is “write”. At decision point 1004, it is determined ifthe event is related to the state of the application. In an embodiment,an event is considered state related if the event has a particular nameand has a desired access of “write,” “set,” or “set value.” Events thatmay be related to state include CreateFile, WriteFile, DeleteFile,CloseFile, RegCreateKey, RegOpenKey, RegSetKeyValue, RegSetValue,RegSaveKey, RegDeleteKey, and RegDeleteValue. In an embodiment, eventssuch as DeleteFile, CloseFile, RegDeleteKey, and RegDeleteValue may berelated to state in that they trigger files/registries to be deletedand/or trigger certain state capture operations as described, forexample, with reference to FIGS. 11A and 11D. If the event is notrelated to the state of the application, then the process proceeds tonormal execution of the event, block 1006. If, however, the event isrelated to the state of the application (e.g., the event has aparticular name and has a desired access of “write,” “set,” or “setvalue”), then the process proceeds to block 1008. At block 1008, theoriginal function of the event is called (e.g., based on staticredirection rules). For example, a function “CreateFile C:\fileC.txt” isredirected as “CreateFile C:\pathA\pathB\fileC.txt”. At decision point1010, it is determined if the function has been successfully completed.If the function has not been successfully completed (e.g., the operatingsystem did not create the file), then the process proceeds to normalexecution, block 1006. If, however, the function has been successfullycompleted, then the process proceeds to block 1012. At block 1012, anotification is sent to the state capture service. For example, thenotification includes information related to the event such as an eventhandle, an event type indicator, an event name, a path, andcorresponding data. Example frame formats for notifications aredescribed below with reference to FIGS. 13-20. As is described in moredetail below, the state capture service helps to capture stateinformation related to the execution of a stateful application on thestateless computing platform. Once the notification is sent, the processproceeds to normal execution, block 1006.

FIGS. 11A-11D illustrate operations that are performed by the statecapture service. At element S, a new instance of an application startsexecution on a stateless computing platform and triggers the launch ofthe state capture service. At block 1102, a location of state relatedinformation in a persistent data store is obtained from the applicationportal. For example, a location within a persistent date store forapplication-specific and user-specific state information is obtainedfrom the application portal. At block 1104, a DomainName/UserName isobtained from the operating system. In an embodiment, obtaining theDomainName/UserName enables execution of the instance of the applicationto be captured for a particular user. For example, the state of anapplication could be captured for two different users, UserA and UserB,with the paths “\\network-share\domain1\UserA\AppX” and“\\network-share\domain1\UserB\AppX”, respectively. When UserA accessesapplication X from the application portal, application-specific anduser-specific state information is obtained from (and saved to) path“\\network-share\domain1\UserA\AppX” and when UserB accesses applicationX from the application portal, application-specific and user-specificstate information is obtained from (and saved to) path“\\network-share\domain1\UserB\AppX”. At block 1106, registries (e.g., a.reg file) are imported from the persistent data store. At block 1108,files based on the configuration file are copied. In an embodiment, aprevious set of storage objects is obtained from the persistent datastore and loaded as a search tree into the in-process memory of thestate capture service. At element “R,” a notification is received fromthe notification engine of the application virtualization layer. Atblock 1110, the notification is read. At decision point 1112, the typeof operation identified by the notification is determined. For example,it is determined if the notification identifies a create/open operation(identified as element “CR”), a write/set operation (identified aselement “WS”), or a close operation (identified as element “CL”). If thenotification identifies a create/open operation (CR), then the processproceeds to FIG. 11B, if the notification identifies a write/setoperation (WS), then the process proceeds to FIG. 11C, and if thenotification identifies a close operation (CL), then the processproceeds to FIG. 11D.

With reference to FIG. 11B, if the notification identifies a create/openoperation (CR), at decision block 1114, it is determined if the storageobject represented in the notification is on an exclusion list. In anembodiment, an exclusion list identifies storage objects that do notrelate to state information or to storage objects that should beexcluded from the state capturing process for some other reason. If thestorage object is on the exclusion list, then the process proceeds backto element R (FIG. 11A) to await another notification. Using anexclusion list may help to prevent storage objects that do not relate tothe state of an application from being added to the set of storageobjects that is monitored by the state capture service. If the storageobject represented in the notification is not on the exclusion list,then the process proceeds to block 1116. At block 1116, the storageobject is recorded in the in-process memory of the state captureservice. For example, the search tree that is maintained in thein-process memory of the state capture service is updated to include thestorage object. Thus, a set of storage objects is automatically learned(e.g., by “machine learning”) during execution of the application in thedeployment phase without the need for human interaction and withouthaving to manually run through each possible workflow in thecontainerization phase. At block 1118, an application-specific anduser-specific redirection rules configuration file (e.g., a rulesconfiguration file) is updated in the application portal and/or thepersistent data store to include the storage object.

With reference to FIG. 11C, if the notification identifies a write/setoperation (WS), at decision point 1120, it is determined if thenotification has an identifier. If the notification does not have anidentifier, then the process proceeds back to element R (FIG. 11A). If,however, the notification does have an identifier, then, at block 1122,a “write” flag is marked for the identifier and the process proceeds toblock 1124. At block 1124, the exact data from the write operation iswritten and then the process proceeds back to element R (FIG. 11A). Inan embodiment, only the delta (e.g., actual data changed) is savedinstead of copying the entire file to the persistent data store everytime there is a write operation. For example, the “WriteFile( )” callcontains a buffer as its second parameter, which is the actual data thatwill be written to the file after executing the operation. In thisexample, the data that was sent in the WriteFile call is read and copiedto the file that exists in the persistent data store. So, instead ofcopying the entire file, just the data that the call holds in itsparameter is written.

With reference to FIG. 11D, if the notification identifies a closenotification (CL), at decision point 1126, it is determined if thenotification has an identifier. If the notification does not have anidentifier, then the process proceeds back to element R (FIG. 11A). If,however, the notification does have an identifier, then, at decisionpoint 1128, it is determined if the “WriteFlag” is marked in thenotification. If the “WriteFlag” is not marked in the notification, thenthe process proceeds back to element R (FIG. 11A). If, however, the“WriteFlag” is marked, then, at decision point 1130, it is determined ifa checksum has changed. If a checksum has not changed (e.g., indicatingthat the data to be written has not changed), then the process proceedsback to element R (FIG. 11A). If, however, the checksum has changed(e.g., indicating that the data to be written has changed), then, atdecision point 1132, it is determined if the notification corresponds toa file or to a registry. If the notification corresponds to a registry,then, at block 1134, the storage object is copied to the corresponding.reg file in the persistent data store. If the notification correspondsto a file, then, at block 1136, the corresponding file is opened in, forexample, shared read mode. Next, at block 1138, the file, or simplechanges in the file, is copied to the persistent data store. Next, atblock 1140, the redirected path, e.g., the original path redirected tothe persistent data store, is recorded in the configuration file on anapplication-specific and user-specific basis.

FIG. 12 is a timing diagram of the workflow described above withreference to FIGS. 1, 2, and 9-11D. At operation 1202, an app containeris uploaded from the containerizer to the application portal. In FIG.12, the application portal includes a persistent data store that is usedto store application state information. At operation 1204, the appcontainer is downloaded from the application portal to a deploymentengine of the stateless computing platform. In an embodiment, thedeployment engine is included in the app container and executes on thestateless computing platform. At operation 1206, the application stateis synchronized with state information that is stored in the persistentdata store. At operation 1208, the application is run (e.g., executed onthe stateless computing platform) using the application virtualizationlayer. Once the application begins running, events are evaluated by theapplication virtualization layer to identify events that may triggerstate changes and notifications are sent to a state capture service whensuch events are identified. At operation 1210, an application statechange event is detected. At the state capture service, thenotifications are evaluated and the set of storage objects is updated inresponse to the evaluations. As described above, machine learning isused to identify state related storage objects during execution of theapplication on the stateless computing platform and the identified staterelated storage objects are added to the set of storage objects that ismonitored. The set of storage objects is stored in the in-process memoryof the state capture service while the application is executing on thestate capture service. The set of storage objects is also stored in thepersistent data store so that the state information is preserved for anext use of the containerized stateful application by the same user.

At operation 1212, the state capture service notifies the applicationvirtualization layer that a state change event has been identified. Atoperation 1214, the application portal is notified of a state changeevent by the application virtualization layer. In an embodiment, theapplication portal updates an application-specific and user-specificconfiguration file in response to the state change event. In anembodiment, a notification to the application portal allows theapplication portal to record a newly learned redirection rule in anapplication-specific and user-specific configuration file so that theredirection rules that are learned during execution of the application(e.g., through the machine learning process) are persisted acrossdifferent execution instances (e.g., execution instance 1 and executioninstance 2) of the containerized application for a particular end-user.For example, if the state capture service learns about a state changingredirection rule that redirects path “P” to path “Q,” (P→Q), then therule is recorded in the in-process memory of the state capture serviceand an event notification, which identifies the redirection rule (e.g.,P→Q), is sent to the application portal. The application portal writesthe new redirection rule in a corresponding application-specific anduser-specific configuration file. When the same application is launchedby the same end-user at a subsequent point in time (as a differentexecution instance and possibly on a different stateless platform), thestate capture service will read the application-specific anduser-specific configuration file and load the file (as its correspondingrules) into its in-process memory. If the state capture serviceencounters path “P” again, path “P” will automatically be redirected topath “Q” without having to learn the redirection rule again. Atoperation 1216, the application virtualization layer notifies the statecapture service that the application is closed (e.g., execution of theapplication at the stateless computing platform is ended) and the statecapture service saves the latest state information, or any last stateupdate, in the persistent data store.

In an embodiment, information related to the events that are generatedby an application is organized into frames for use in the operationsdescribed above. Example frame formats are described with reference toFIGS. 13-20. FIG. 13 is an example of a generic frame format used forrecording state change operations. The frame format includes a handlefield, a type field, an operation field, a location field, and a datafield. In an embodiment, the handle field identifies a handle, which isa unique identifier to identify a file/registry's record within thememory of the state capture service. The type field identifies a Booleanflag that indicates if the corresponding operation is file related orregistry related. For example, the flag can be ‘0’ for file and ‘1’ forregistry. The operation field specifies what operation, e.g., API call,was triggered. The path field is an optional field that specifies thefull path of the file/registry associated with the operation. Forexample, the path field is typically used in create/open operationnotifications. The data field is an optional field specifying thedata/value that is to be written, set, or modified. For example, thedata field is typically used in write/set operation notifications.

FIG. 14 is an example of a create file/open file frame format. Thecreate file/open file frame format includes an identifier field, a typefield, an API call field, and a path field. In an embodiment, theidentifier field is the local file handle with which the file wascreated/opened. Though this handle is valid only within the applicationwhich is triggering the operation, the handle serves as a uniqueidentifier within the memory of the state capture service. The typefield for a CreateFile operation is File, which is indicated by 0. TheAPI call field is CreateFile as that is the corresponding operationperformed by the application. The path field for the CreateFileoperation is a mandatory field, which includes the entire path of thefile being created/opened (e.g., C:\TEMP\A.txt).

FIG. 15 is an example of a write file frame format. The write file frameformat includes an identifier field, a type field, an API call field,and a data field. In an embodiment, the identifier field is the localfile handle for the file. The type field for a WriteFile operation isFile, which is indicated by 0. The API call field is WriteFile as thatis the corresponding operation performed by the application. The datafield for the WriteFile operation is the data to be written.

FIG. 16 is an example of a close file frame format. The close file frameformat includes an identifier field, a type field, and an API callfield. In an embodiment, the identifier field is the local file handlefor the file. The type field for a CloseFile operation is File, whichindicated by 0. The API call field is CloseFile as that is thecorresponding operation performed by the application.

FIG. 17 is an example of a create registry frame format. The createregistry frame format includes an identifier field, a type field, an APIcall field, and a path field. In an embodiment, the identifier field isthe local registry handle with which the registry was created. The typefield for a create registry operation is Reg, which is indicated by 1.The API call field is RegCreateKey as that is the correspondingoperation performed by the application. The path field for theRegCreateKey operation is a mandatory field, which includes the entirepath of the registry being created (e.g., HKCU\XX\YY).

FIG. 18 is an example of an open registry frame format. The openregistry frame format includes an identifier field, a type field, an APIcall field, and a path field. In an embodiment, the identifier field isthe local registry handle with which the registry was opened. The typefield for an open registry operation is Reg, which is indicated by 1.The API call field is RegOpenKey as that is the corresponding operationperformed by the application. The path field for the RegOpenKeyoperation is a mandatory field, which includes the entire path of theregistry being opened (e.g., HKCU\XX\YY).

FIG. 19 is an example of a modify registry frame format. The modifyregistry frame format includes an identifier field, a type field, an APIcall field, and a value field. In an embodiment, the identifier field isthe local registry handle. The type field for a modify registryoperation is Reg, which is indicated by 1. The API call field isRegSetValue as that is the corresponding operation performed by theapplication. The value field for the RegSetValue operation is value thatis to be set.

FIG. 20 is an example of a close registry frame format. The closeregistry frame format includes an identifier field, a type field, and anAPI call field. In an embodiment, the identifier field is the localregistry handle for the registry. The type field for a close registryoperation is Reg, which is indicated by 1. The API call field isRegCloseKey as that is the corresponding operation performed by theapplication.

Various operations utilizing the above-described frame formats are nowdescribed. Consider a scenario where a containerized application iswriting to a file. If the dynamic redirections approach is used, the APIcalls are recorded in the in-process memory of the applicationvirtualization layer and the application virtualization layer performsthe activities to capture the state of the application. Whereas, if thestate capture service approach is used, then the applicationvirtualization layer sends a notification to state capture service andthe state capture service performs the rest of the activities related tocapturing the state of application.

In an example operation, an application will first perform a“create/open file” operation with desired access of “write” flag(FILE_APPEND_DATA, FILE_WRITE_ATTRIBUTES, FILE_WRITE_DATA,FILE_WRITE_EA, STANDARD_RIGHTS_WRITE, SYNCHRONIZE). The applicationvirtualization layer will get a hold of this call from within itshandlers and detect that the application is likely to perform a writeoperation on the file. The application virtualization layer will thenuse CreateFile/OpenFile format (see FIG. FIG. 14) to track the state ofthe application based on which approach is used (dynamic redirections orstate capture service).

The state capture service will record this operation in its in-processmemory along with the information about the operation. After thecreate/open file, the application executes a WriteFile call to modifythe contents of the file. The application virtualization layer will geta hold of this call from within handlers and extract the informationsuch as the file handle. The application virtualization layer mightoptionally extract the contents from data getting written. Then, theapplication virtualization layer will either send a notification tostate capture service (as shown in FIG. 15) or record the frame in itsown in-process memory. When using the dynamic redirections approach, theapplication virtualization layer will redirect the call to thepersistent data store and in the case of the state capture serviceapproach, the state capture service will mark its in-memory recordrelated to the file handle as “Written.” Marking the file as “Written”indicates that the file is to be captured in the persistent data store(as it has been modified). Then, the state capture service can eithercopy the whole file to the persistent data store or append the data fromthe optional data field to the file (e.g., depending on the file pointerlocation). The state capture service approach can provide a benefit ofwriting only what has changed (e.g., the “delta”) instead of copying thewhole file.

After some time, the application will eventually close the file. Whenthe application virtualization layer receives a close call (e.g., fromwithin its handler, such as NtCloseFile), the application virtualizationlayer performs the closing operations. If the dynamic redirectionsapproach is used, then the application virtualization layer will justredirect the call to close the file in the persistent data store.Whereas, if the state capturing service approach is used, theapplication virtualization layer will send a close notification to statecapture service using the close file notification frame format as shownin FIG. 16. The state capture service will then mark its in-processmemory record related to this file handle as “Closed.” After the actualapplication finishes the close file operation, the state capture servicewill either copy the whole file from its original location to thepersistent data store by opening the file in shared mode (e.g., so thatother applications do not get locked), or the state capture service canperform a delta write using the data that was sent in the WriteFilenotification.

In an embodiment, the processing of registry related events is similarto the processing of file related events. If the dynamic redirectionsapproach is used, the API calls will be recorded in applicationvirtualization layer's in-process memory and the applicationvirtualization layer performs the activities to capture the state of theapplication's registry. Whereas, if the state capturing service approachis used, then the application virtualization layer will send anotification to state capturing service and the state capturing servicewill perform the activities of capturing the state of application.

In the case of registry related operations, it is likely that theapplication will first perform a “create/open registry” operation withdesired access of “write” or “set value.” The application virtualizationlayer will get a hold of the API call from within its handlers anddetect that the application is likely to perform a write operation onthis registry key or subkeys. The application virtualization layer willthen use the CreateRegistry or OpenRegistry frame format as describedwith reference to FIGS. 17 and 18 to track the state of the applicationbased on which approach is used, dynamic redirections or state captureservice).

After the create/open registry, the application executes a registryrelated call such as RegSetValue, RegSetKeyValue, RegDeleteValue,RegDeleteKeyValue to modify the contents of the registry. Theapplication virtualization layer will get a hold of this call fromwithin ntdll handlers and extract information such as the registryhandle. The application virtualization layer might optionally extractthe contents from data getting set for a SetValue type of operation.Then, the application virtualization layer will either send anotification to state capture service, such as the notificationdescribed with reference to FIG. 19, or record the value in its ownin-process memory. In the case of the dynamic redirections approach, theapplication virtualization layer will redirect the call to thepersistent data store and in the case of the state capture serviceapproach, the state capture service will mark its in-process memoryrecord related to this registry handle as “Written.” Marking the recordas “Written” indicates that the registry is to be captured in thepersistent data store (as it has been modified). Then, the state captureservice can either copy the whole registry hive to the persistent datastore or set the data from the optional data field to the registry(depending on the file pointer location). The second approach mayprovide a benefit of setting only the value of what has changed insteadof copying the whole registry. If the delete calls is triggered, thenthe corresponding key is deleted from persistent storage and theapplication virtualization layer/state capture service removes the keyand/or all entries corresponding to the key/value from its memory.

After few set/query calls, the application will eventually close theregistry handle. When the application virtualization layer receives aclose call (from within its handlers), the application virtualizationlayer performs the closing operations. If the dynamic redirectionsapproach is used, then the application virtualization layer will justredirect the call to close the registry in the persistent data store.Whereas, if the state capturing service approach is used, then theapplication virtualization layer will send a close notification to statecapture service using a close registry notification frame format, suchas the frame format described with reference to FIG. 20. The statecapture service will then mark its in-process memory record related tothis registry handle as “Closed.” After the application finishes theclose registry operation, the state capture service will either copy thewhole registry hive from the original location to the persistent datastore by opening the registry hive in shared mode (so that other appsdon't get locked), or the state capture service can perform a deltawrite using the data that was sent in the SetFile notification.

FIG. 21 is a process flow diagram of a method for executing acontainerized stateful application that is deployed on a statelesscomputing platform. At block 2102, a containerized stateful applicationis deployed on a stateless computing platform. At block 2104, thestateful application executes on the stateless computing platform. Atblock, 2106, during execution of the stateful application, events thatare generated during execution of the stateful application areevaluated, in an application virtualization layer, to identify eventsthat may trigger a change in state of the stateful application. At block2108, during execution of the stateful application, a set of storageobjects is updated in response to the evaluations. At block 2110, duringexecution of the stateful application, events that are generated by thestateful application are compared to the set of storage objects and astorage object that corresponds to an event is redirected to apersistent data store if the storage object matches a storage object inthe set of storage objects.

Although the operations of the method(s) herein are shown and describedin a particular order, the order of the operations of each method may bealtered so that certain operations may be performed in an inverse orderor so that certain operations may be performed, at least in part,concurrently with other operations. In another embodiment, instructionsor sub-operations of distinct operations may be implemented in anintermittent and/or alternating manner.

It should also be noted that at least some of the operations for themethods may be implemented using software instructions stored on anon-transitory computer useable storage medium for execution by acomputer. As an example, an embodiment of a computer program productincludes a computer useable storage medium to store a computer readableprogram that, when executed on a computer, causes the computer toperform operations, as described herein.

Furthermore, embodiments of at least portions of the invention can takethe form of a computer program product accessible from a computer-usableor non-transitory computer-readable medium providing computer executableinstructions, or program code, for use by or in connection with acomputer or any instruction execution system. For the purposes of thisdescription, a non-transitory computer-usable or computer readablemedium can be any apparatus that can contain or store the program foruse by or in connection with the instruction execution system,apparatus, or device.

The computer-useable or computer-readable medium can be an electronic,magnetic, optical, electromagnetic, infrared, or semiconductor system(or apparatus or device). Examples of a computer-readable medium includea semiconductor or solid state memory, magnetic tape, a removablecomputer diskette, RAM, ROM, a rigid magnetic disk, and an optical disk.Current examples of optical disks include a compact disk with read onlymemory (CD-ROM), a compact disk with read/write (CD-R/W), and a digitalvideo disk (DVD).

In the above description, specific details of various embodiments areprovided. However, some embodiments may be practiced with less than allof these specific details. In other instances, certain methods,procedures, components, structures, and/or functions are described in nomore detail than to enable the various embodiments of the invention, forthe sake of brevity and clarity.

Although specific embodiments of the invention have been described andillustrated, the invention is not to be limited to the specific forms orarrangements of parts so described and illustrated. The scope of theinvention is to be defined by the claims appended hereto and theirequivalents.

In the above description, specific details of various embodiments areprovided. However, some embodiments may be practiced with less than allof these specific details. In other instances, certain methods,procedures, components, structures, and/or functions are described in nomore detail than to enable the various embodiments of the invention, forthe sake of brevity and clarity.

Although specific embodiments of the invention have been described andillustrated, the invention is not to be limited to the specific forms orarrangements of parts so described and illustrated. The scope of theinvention is to be defined by the claims appended hereto and theirequivalents.

What is claimed is:
 1. A method for executing a containerized statefulapplication that is deployed on a stateless computing platform, themethod comprising: deploying a containerized stateful application on astateless computing platform; executing the stateful application on thestateless computing platform; during execution of the statefulapplication, evaluating, in an application virtualization layer, eventsthat are generated during execution of the stateful application toidentify events that may trigger a change in state of the statefulapplication; during execution of the stateful application, updating aset of storage objects in response to the evaluations; and duringexecution of the stateful application, comparing events that aregenerated by the stateful application to the set of storage objects andredirecting a storage object that corresponds to an event to apersistent data store if the storage object matches a storage object inthe set of storage objects.
 2. The method of claim 1, wherein updatingthe set of storage objects comprises adding a storage object to the setof storage objects if an event is identified as an event that maytrigger a change in state of the stateful application and if the storageobject is not already in the set of storage objects.
 3. The method ofclaim 2 wherein the set of storage objects is updated in in-processmemory of application virtualization layer and in the persistent datastore.
 4. The method of claim 2 wherein the set of storage objects isupdated in in-process memory of a state capture service and in thepersistent data store.
 5. The method of claim 2, wherein events that maytrigger a change in state of the stateful application are events from apredefined set of events.
 6. The method of claim 5, wherein thepredefined set of events includes file-related events.
 7. The method ofclaim 6, wherein the file-related events include CreateFile, WriteFile,DeleteFile, and CloseFile.
 8. The method of claim 2, wherein thepredefined set of events includes registry-related events.
 9. The methodof claim 8, wherein the registry-related events include RegCreateKey,RegOpenKey, RegSetKeyValue, RegSetValue, RegSaveKey, RegDeleteKey, andRegDeleteValue.
 10. The method of claim 2, wherein events that maytrigger a change in state of the stateful application include eventsthat have a desired access of write.
 11. The method of claim 2, whereinevents that may trigger a change in state of the stateful applicationinclude file-related events that have a desired access of write.
 12. Themethod of claim 2, wherein the events that may trigger a change in stateof the stateful application include events that have a permission ofset.
 13. The method of claim 2, wherein the events that may trigger achange in state of the stateful application include registry-relatedevents that have a permission of set.
 14. The method of claim 1, whereinupdating the set of storage objects comprises updating the set ofstorage objects in in-process memory of the stateless platform.
 15. Themethod of claim 1, wherein updating the set of storage objects comprisesupdating the set of storage objects in the persistent data store.
 16. Anon-transitory computer readable medium that stores computer-executablecode, which when executed by one or more processors implements a method,the method comprising: deploying a containerized stateful application ona stateless computing platform; executing the stateful application onthe stateless computing platform; during execution of the statefulapplication, evaluating, in an application virtualization layer, eventsthat are generated during execution of the stateful application toidentify events that may trigger a change in state of the statefulapplication; during execution of the stateful application, updating aset of storage objects in response to the evaluations; and duringexecution of the stateful application, comparing events that aregenerated by the stateful application to the set of storage objects andredirecting a storage object that corresponds to an event to apersistent data store if the storage object matches a storage object inthe set of storage objects.
 17. The non-transitory computer readablemedium of claim 16 that stores computer-executable code, which whenexecuted by one or more processors implements a method, wherein updatingthe set of storage objects comprises adding a storage object to the setof storage objects if an event is identified as an event that maytrigger a change in state of the stateful application and if the storageobject is not already in the set of storage objects.
 18. Thenon-transitory computer readable medium of claim 17 that storescomputer-executable code, which when executed by one or more processorsimplements a method, wherein the set of storage objects is updated inin-process memory of application virtualization layer and in thepersistent data store.
 19. The non-transitory computer readable mediumof claim 17 that stores computer-executable code, which when executed byone or more processors implements a method, wherein the set of storageobjects is updated in in-process memory of a state capture service andin the persistent data store.
 20. The non-transitory computer readablemedium of claim 16 that stores computer-executable code, which whenexecuted by one or more processors implements a method, wherein eventsthat may trigger a change in state of the stateful application includeevents that have a desired access of write or set.